1. Field of the Invention
The present invention relates generally to hardware and software testing and verification, and specifically to testing software on low-end emulators and computing devices.
2. Description of the Related Art
The meanings of acronyms and certain terminology used herein are given in Table 1:
TABLE 1APIApplication programming interfaceCLDCConnected, limited device configuration. CLDC issuitable for devices with 16/32-bit RISC/CISCmicroprocessors/controllers, having as little as160 KB of total memory available.HTTPHyperText Transfer ProtocolIDIdentifierIPInternet ProtocolJ2EEJava 2 Enterprise EditionJ2MEJava 2 Micro EditionJ2SEJava 2 Standard EditionJADJava application descriptorJARJava archiveJDTSJava Device Test Suite Execution FrameworkMIDletA MIDP applicationMIDPMobile information device profile. A set of JavaAPIs, which, together with the CLDC, provides acomplete J2ME application runtime environmenttargeted at mobile information devices.
MIDP is defined in Mobile Information Device Profile (JSR-37), JCP Specification, Java 2 Platform, Micro Edition, 1.0a (Sun Microsystems Inc., Palo Alto, Calif., December 2000). MIDP builds on the Connected Limited Device Configuration (CLDC) of the Java 2 Platform, Micro Edition (J2ME) (available from Sun Microsystems Inc., Palo Alto, Calif.). The terms Sun, Sun Microsystems, Java, J2EE, J2ME, J2SE, and the Sun logo are trademarks or registered trademarks of Sun Microsystems, Inc., in the United States of America and other countries. All other company and product names may be trademarks of their respective companies. A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by any one of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
Tools have been developed in recent years to aid in the design verification of hardware and software systems, for example software suites, hardware circuitry, and programmable logic designs. In order to assure that the design complies with its specifications, it is common to generate a large number of input or instruction sequences to assure that the design operates as intended under a wide variety of circumstances. In general, test systems produce a report indicating whether tests have been passed or failed, and, in some cases may even indicate a module that is estimated to be faulty.
Conventionally, in order to test a device under development (such as a mobile information device), or to test software designed to run on such a device, a developer connects the device to an appropriate test system. The target device under test may be connected to the test system either directly or via a communication emulator. The developer selects a battery of test programs to run on the target device while monitoring its behavior. Running the complete battery of tests can commonly take many hours or even days. This problem is particularly acute in testing low-end computing devices, such as cellular telephones and other mobile information devices, which have limited computing power and memory resources. Thus, testing on the target device can become a serious bottleneck in the development cycle.
A centralized system for centrally managing test suites is disclosed in commonly assigned application Ser. No. 10/767,849, entitled “Automated Test Execution Framework with Central Management”, which is herein incorporated by reference. In this arrangement, a central repository contains a management unit, available test suites and a single test execution harness or framework. Using the management unit, a system administrator establishes active versions of the various test suites and their individual configurations. End users install clients of the central repository, using a system-provided installer program. In each client, an execution script is created, which downloads the harness and a local configuration file. Then, when the harness is executed at the client, it loads with all designated test suites already installed, configured and ready for execution. The client always has the most current versions of all test suites. Advantageously, all necessary information is obtained from a single central location.
A further improvement in test suite management is disclosed in commonly assigned application Ser. No. 10/767,850, entitled “Parallel Test Execution on Low-End Emulators and Devices”, which is herein incorporated by reference. While this arrangement facilitates testing large numbers of devices simultaneously, it does not optimally reduce the time required for the verification phase of the development cycle. This is particularly the case where a developer wishes to exhaustively test a device or software program using a very large number of different tests.